Computer systems rely on the cooperative interaction of numerous, specialized devices configured to perform a variety of tasks. Typically, the devices interact by reading or writing data to other components in the system. The successful exchange of data between these components is consequently vital to the operation of computer systems. Such devices commonly include a processor, memory and certain peripheral devices, such as mass storage devices, network adapters, display adapters, audio adapters and workstation controllers. These devices are typically coupled together using a network of address, data and control lines, commonly referred to as a bus.
One common type of bus is known as the Peripheral Component Interconnect (PCI) bus. Under the PCI bus standard, peripheral components can connect to a PCI bus without the need for glue logic. Thus, PCI provides a bus standard on which high performance peripheral devices, such as graphics devices, control panels, tape drives, as well as optical and hard disk drives, can be coupled to a processor. Bus standards more particularly refer to an independent set of protocols, or rules, to conduct data transfers between the various devices attached to it. Each of these protocols is designed into a bus directly and is commonly referred to as the architecture of the bus.
In a data transfer between different bus architectures, data being transferred from the first bus architecture may not be in a form that is usable or intelligible by the receiving second bus architecture. Accordingly, mechanisms have developed for translating data that is required to be transferred from one bus architecture to another. This translation mechanism is normally contained in a hardware device in the form of a bus-to-bus bridge, or interface, through which the two different types of buses are connected.
Various bus-to-bus bridges have consequently been designed to match the communication protocol of one bus with that of another. These bridges thus permit system wide communications between devices on different buses. For example, a bus-to-bus bridge connecting between a system bus and a PCI local bus is called a PCI host bridge. The PCI host bridge contains all the logic and hardware for translating data communications between the system bus and the PCI local bus, and ensures that data is transferred between these two buses intelligibly.
A variant of PCI, PCIX, includes a host bridge through which the respective devices may gain access to the bus. The host bridge performs write back splits, or communicates with the system bus on behalf of requesting devices. For instance, an Ethernet device may send a data request to a host bridge, which in turn, sends a request for the data to a bus coupled to a memory device storing the data. The bridge then receives the data from the memory device and sends it back to the Ethernet device. Each bridge usually has posting buffers for temporary buffering of bus transactions, as these transactions flow through the bridge in both directions.
Multiple devices connected to the different buses must not be allowed access via the host bridge to the processor or a local bus at the same time. Such simultaneous access would confuse computer systems, producing unuseable results. Such confusion is avoided by using a bus arbiter. A bus arbiter controls device access to the bus. Processes used to decide when a device may next access the bus via the host bridge is appropriately called bus arbitration. In a bus arbitration scheme, a device (that may include the host bridge) wanting to use the bus signals a bus request. In response, the arbiter sends a grant signal to the device. After the grant is received, the device may send a request to the host bridge, prompting the host bridge to access the bus on the other side of the bridge on behalf of the device, i.e., to read or write data according to the request. The arbiter can then grant to another device the privilege of having cycles run by the host bridge on its behalf.
Arbitration schemes are usually designed to balance two factors when determining a sequence for granting the bus via the host bridge. First, each device request typically has a bus priority, and the highest priority devices are serviced first. Second, to help avoid instances where low priority devices are locked out, most I/O buses such as PCI also require the arbiter to implement some kind of fairness protocol. The intent of a fairness protocol is to assure that all devices receive a turn on the bus. For instance, one conventional fairness protocol is a round robin scheme. Under a round robin fairness protocol, a device that has just completed a bus operation is not granted access to the bus for a second operation until all other requesting devices have first been granted access to the bus.
Even though a bus may provide a fairness protocol in the arbiter(s) that control access to the bus, acceptable access to the bus can be effectively denied, or locked, to devices. If several or all of the devices are competing for a single resource, such as system memory, and one of the devices is monopolizing the memory, then another device needing access to the memory may be unable to access the monopolized resource. This lockout may occur even though the arbiter fairly grants bus access to all requesting devices. The locked out device is thus unable to capitalize on its bus access allowed by the arbiter, and consequently receives a retry signal, relegating the device to attempting the transaction at some later time.
More particularly, many buses provide a performance feature usually referred to as retry capability that allows a device to disconnect from the bus/host bridge if it is not able to handle the request at that time. If a target device is not able to handle a request when it occurs, that target can issue a retry, which indicates to the device that issued the request on the bus to try again later.
Lockout generally involves interaction between the set of buffers in a bridge, the arbiter, and the bus traffic by devices on the bus. For example, an arbiter may have a round-robin fairness protocol for five devices (Device A, Device B, Device C, Device D and a host bridge). The host bridge is assigned the highest priority (priority 0), Device A is assigned the next highest priority (priority 1), Device B is assigned the next highest priority (priority 2), Device C is assigned the next highest priority (priority 3), and Device D is assigned the next highest priority (priority 4). If all devices ask for the bus at the same time, the fairness protocol will assure that each device gets a chance to try to utilize the bus. The arbitration priority, in this example, simply determines the order in which the devices get a turn to try to utilize the bus.
When all devices request use of the bus, the host bridge is granted first access, then Devices A to D in sequence. In one scenario under this scheme, Device A may be running a large amount of reads to system memory on the other side of the host bridge. Device A will consequently re-request the bus for a new transaction as soon as it completes the current transaction. As such, the host bridge contains four read buffers that are all currently allocated to Device A reads. Device B may subsequently want to do some reads. When Device B gets the bus, however, Device B gets a retry because the bridge's buffers are full with Device A reads. That is, while the host bridge is acquiring the read data for its four buffers, all other devices that run reads to the host bridge will receive retries. Eventually, the bridge empties out one of its buffers onto the bus, completing one of the read transactions to Device A. The host bridge now has one of its four buffers free, but because in the current scheme Device A will always get the bus after the host bridge gets the bus because Device A is always requesting the bus. Device A will consequently get the bus next and queue another read to the host bridge, using the one free buffer slot. Next, the arbiter will grant the bus to Device B, but again, there will be no buffers free to usem and Device B will receive a retry.
Because the host bridge must get a turn on the bus in order to free up a buffer, and because Device A always gets to run right after the host bridge, it is clear that Device A will continue to be able to claim every buffer that frees up, locking out Device B for as long as Device A wants to do reads. A lockout can occur such that each time a specific device gets a turn on the bus and is turned away with a retry (or equivalent, depending on the bus type) because other devices keep filling up the bridge buffers, i.e., saturating the host bridge. A large number of retries could result in significant performance losses or even errors for the device that is being locked out. For example, the device being locked out may be an ethernet controller that has a requirement to deliver the next packet within a certain period of time, or else the packet will time out and be considered lost. The ethernet controller cannot deliver the packet in time because it is being locked out from reading the packet's data from system memory.
The arbiter does not know or care that there are two devices competing for a common resource. The arbiter does not know the destination of the various transactions and how they interrelate. The arbiter is unaware of which devices are retried. As far as the arbiter is concerned, the transaction was successful in that the device was granted access to run on the bus, even though the device was unable to receive the requested data. The arbiter has fulfilled its programmed fairness requirement.
Therefore, what is needed is an improved system for arbitrating device access to a desired bus.